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Abstract-  We  present  a  methodology  for  the  creation, 
manipulation  and  transmission  of  3D  anatomical  models 
starting  from  stacks  of  medical  images.  The  anatomical 
information  for  the  models  is  first  segmented  from  the 
images  and  then  used  by  a  surface  reconstruction 
algorithm  to  create  3D  meshes  that  accurately  represent 
the  surface  of  the  organs  being  modeled.  The  meshes  are 
then  exported  in  an  open  format  for  manipulation  in  a 
graphics  platform.  Using  high-end  graphics  algorithms 
we  render  and  texture-map  our  meshes.  We  can  also 
deform  the  meshes  and  assign  physical  and  material 
properties  to  them.  Our  models  can  be  used  for  virtual 
reality  applications  and  can  also  be  transmitted  over  the 
Internet  for  remote  interactive  visualization  and  retrieval. 
Keywords  -  3D  Models,  Java,  VRML,  X3D,  Medical  Web 
Applications,  Medical  Graphics,  Visualization,  Telemedicine 

I.  Introduction 

In  the  past  decades  medical  imaging  systems  such  as  CT  and 
MRI  have  become  widely  used  diagnostic  tools  for  clinical 
diagnosis  and  medical  scientific  research.  However,  as 
household  as  these  technologies  have  become,  they  remain 
expensive  systems  readily  available  only  in  radiology  clinics. 
It  is  until  recently,  that  the  Internet,  combined  with  powerful 
PC’s,  new  graphics  standards  and  web-based  technologies 
have  made  previously  restricted  MR/CTI  data  available  to 
everyone,  even  modest  hospitals  in  small  communities  or 
rural  areas.  The  Visible  Human  Project  [1]  was  a  landmark 
for  widespread  availability  of  radiological  data.  Current 
Internet  2  technology,  combined  with  new  specifications  for 
the  transmission  of  graphical  data  such  as  X3D  and  Lattice 
[2]  [3],  offer  promises  that,  in  the  short  to  medium  term,  not 
only  images,  but  full  3D  models,  and  even  virtual  reality 
scenarios  and  3D  medical  simulators  will  be  available  over 
the  Internet.  Some  authors  [4]  [5],  have  already  used  web- 
based  languages  such  as  VRML  and  Java3D  to  create  Internet 
available  medical  applications. 

In  this  paper  we  present  a  methodology  for  the  creation  of 
accurate,  realistic  3D  anatomical  models  starting  from  a  stack 
of  2D  images.  We  use  robust  segmentation  and  reconstruction 
methods  that  have  been  developed  in  our  lab  in  the  past  years 
[6]  [7]  to  go  from  2D  images  to  3D  meshes  corresponding  to 
the  anatomical  models.  The  anatomical  models  created  can  be 
exported  in  open  format  for  its  use  in  most  graphical,  CAD  or 
modeling  software.  We  are  also  able  to  transmit  the  models 
over  the  Internet  using  Java,  VRML,  and  X3D  technologies 
for  remote  users  to  visualize,  manipulate  and  retrieve  them 
for  local  use. 


II.  Methodology 
A.  Segmentation 

We  start  the  creation  of  our  3D  models  by  segmenting  a 
stack  of  2D  images.  All  of  our  models  have  been  created  by 
segmenting  stacks  of  MR  studies.  Our  segmentation 
algorithm  identifies  all  the  pixels  from  a  particular  anatomical 
structure  inside  an  MR  image.  We  work  with  information  of  2 
images  at  a  time  and  assume  that  each  distinct  anatomical 
structure  is  represented  by  a  relatively  uniform  gray  level. 
With  this  simplified  model  of  the  MR  images  we  achieve  a 
very  fast,  practically  automatic  segmentation  tool  with  only 
one  parameter  to  adjust.  The  automatic  segmentation  of  a 
stack  of  over  100  MR  images  of  the  brain  is  done  in  under  1 
minute  in  a  PHI  PC.  When  the  segmentation  is  interactively 
guided  to  achieve  improved  results,  the  visual  interface  that 
modifies  the  algorithm  parameter  allows  the  user  to  quickly 
improve  the  results  and,  in  this  case,  the  segmentation  of  a 
whole  stack  of  128  images  takes  roughly  between  10-15 
minutes  when  performed  by  an  experienced  user.  A  detailed 
description  of  the  segmentation  algorithm  can  be  found  in  [6]. 
An  example  of  the  segmentation  results  used  to  build  the 
anatomical  models  presented  in  this  paper  is  shown  in  Fig.l. 


Fig.  1 .  Segmentation  results  of  3  non  consecutive  images.  The  original  image 
is  shown  in  the  1st  column.  Segmentation  of  the  white  matter,  cerebellum  and 
left  and  right  ventricles  are  shown  in  the  2nd  and  3rd  columns. 
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B.  Reconstruction 


We  use  a  very  robust  3D  surface  reconstruction  algorithm 
developed  in  our  lab  and  described  in  detail  in  [7]  [8].  The 
reconstruction  algorithm  takes  as  input  the  set  of  3D  points 
resulting  from  the  segmentation  of  the  stack  of  images  and 
produces  a  3D  triangular  mesh  that  closely  approximates  the 
real  surface  of  the  models.  The  reconstruction  algorithm  uses 
space  partitioning,  filtering  and  dynamical  rules  to  build  a 
detailed  final  mesh.  The  density  of  the  mesh  can  be 
interactively  adjusted  by  the  user  depending  on  the  density  of 
the  set  of  points  resulting  from  the  segmentation  and  on  the 
complexity  of  the  surface  to  be  reconstructed.  This  mesh  can 
then  be  furthered  simplified  to  eliminate  triangles  in  areas  of 
less  detail  to  create  compressed  models  that  can  then  be  used 
for  remote  and  web-based  applications.  The  details  of  the 
simplification  algorithm  are  described  in  [9].  Some 
reconstruction  examples  are  shown  in  Figs  2,3.  The  first 
column  in  Fig.  2  shows:  a)  the  set  of  points  produced  after 
segmenting  the  cerebellum  and  ventricles  from  an  MR  study 
of  the  brain,  b)  the  3D  mesh  that  reconstructs  the  surface  of 
the  set  of  points,  and  c)  the  rendered  mesh.  The  second 
column  of  Fig.  2  shows  the  same  results  for  the  left  and  right 
ventricles.  Fig.  3  shows  3  dense  meshes  corresponding  to  2 
pairs  of  ventricles  and  the  white  matter  and  different  stages  of 
mesh  compression. 


Fig.  2.  Reconstruction  of  the  cerebellum  with  ventricles  attached  and  of  the 
left  and  right  ventricles  alone. 


Fig.  3.  3D  meshes  corresponding  to  the  ventricles  and  white  matter  with 
3  different  levels  of  mesh  compression. 

C.  Graphical  Production  and  Manipulation 

The  3D  meshes  that  result  from  the  reconstruction  process 
can  be  output  in  DXF  or  VRML  format,  making  them 
suitable  for  manipulation  by  graphics  packages.  We  use 
Maya™ ,  a  high-end  graphics  software  to  render  our  3D 
meshes  with  various  material  and  lightning  properties.  We  are 
also  able  to  give  our  meshes  physical  properties  to  simulate 
dynamic  phenomena.  An  interesting  option  in  Maya  is  the 
possibility  to  interactively  deform  and  render  the  meshes  for 
applications  such  as  cosmetic  and  reconstructive  surgery  or 
for  matching  and  registration  of  information.  We  are  also  able 
to  construct  complex  scenes  containing  several  anatomical 
models  as  needed  for  the  construction  of  atlases  or  virtual 
scenarios.  Because  of  the  physical  properties  of  the  models, 
they  can  interact  with  each  other  in  a  realistic  way  to  simulate 
simplified  surgical  procedures  or  collisions  as  a  result  of  an 
accident. 


D.  Web-based  Application 


The  3D  anatomical  models  can  also  be  transmitted  over 
the  Internet  for  remote  interactive  visualization  and  retrieval. 

We  use  a  VRML  representation  of  the  3D  meshes  where 
we  encode  the  information  about  vertices,  triangles,  normals, 
colors  and  texture  coordinates.  Using  JavaScript  we  create  a 
user  interface  for  the  user  to  interactively  visualize  the 
meshes  and  download  the  information  for  local  use.  Our 
current  work  includes  integrating  the  recent  graphics 
specification  X3D  and  its  Lattice  extension  (as  proposed  at 
the  Web3D  Consortium)  to  accelerate  the  transmission  rate  of 
our  models  in  low  band- width  networks. 

III.  Results 

Some  resulting  anatomical  models  after  rendering  and 
texture  mapping  are  shown  in  Fig.  4.  Again,  these  results 
were  produced  from  the  segmentation  and  reconstruction  of 
MR  images.  We  show  the  model  of  the  left  and  right 
ventricles,  of  the  skull  and  of  the  white  matter.  In  Fig. 5  we 
show  the  possibility  to  render  some  models  as  semi¬ 
transparent  in  order  to  combine  models  and  create  anatomical 
atlases.  We  show  the  skull  rendered  with  transparency  and 
containing  the  ventricles  inside.  Fig.  6  shows  a  graphical 
manipulation  of  the  models  consisting  of  cutting  the  white 
matter  in  half  to  be  able  to  analyse  it  from  the  inside  as  well 
as  the  outside  of  the  model.  This  type  of  visualisation  can  be 
important  when  rendering  artifacts  create  some  ambiguity  on 
the  information  that  cannot  be  resolved  by  analysing  just  one 
side  of  the  surface. 


Fig.  4.  Anatomical  models  corresponding  to  the  ventricles,  the  skull  and 
the  white  matter  after  rendering  and  texture  mapping. 


Fig.  5.  Models  rendered  using  transparency  and  superposition.  We  see  a 
transparent  white  matter  containing  the  ventricles  inside. 


Maya™  a  high  end  graphics  software  to  manipulate  the 
models  in  a  variety  of  ways:  from  simple  rendering  and 
texture  mapping,  to  deformation,  simulation  of  collisions  and 
creation  of  complex  virtual  scenes  containing  multiple 
objects.  We  concentrate  our  current  work  in  creating  an 
efficient  web-based  application  for  the  transmission  of  the 
models.  We  use  VRML  descriptions  of  the  3D  anatomical 
models  and  JavaScript  to  create  a  web-user  interface  where 
the  user  can  interactively  visualize  and  retrieve  the  models 
over  the  Internet.  We  intend  to  migrate  our  model  description 
to  X3D  in  order  to  speed  up  the  model  transmission  over  low- 
band  networks.  Our  application  fits  in  various  fields  of  use: 
from  medical  anatomical  training,  to  the  creation  of  medical 
virtual  scenarios,  graphics  applications,  clinical  diagnosis  and 
telemedicine. 
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IV.  Discussion 

For  some  years  now,  much  work  has  been  devoted  to  the 
creation  of  static  3D  anatomical  models  and  at  least  as  much 
work  to  the  simulation  of  deformations  on  the  models.  The 
possibility  of  creating  3D  anatomical  computer  models  as 
substitutes  of  real  organs  has  seen  big  advances,  but  it  is  until 
recent  years,  that  powerful  graphical  software  and  hardware 
driven  mainly  by  the  entertainment,  games  and  animation 
industry  have  made  it  possible  to  have  realistic  medical 
simulators  and  medical  virtual  scenarios  at  our  desktop.  More 
and  more,  we  are  prone  to  encounter  such  graphical  tools  at 
the  doctors  office,  but  the  possibilities  of  these  new 
applications  remain  wide  open  and,  most  important,  remain  to 
be  made  available  to  a  large  audience  of  potential  users. 

Web-based  applications  make  it  possible  to  bring  new  and 
exciting  graphics  technology  to  remote  locations  for  uses  in 
all  areas  of  Telemedicine.  However,  transmission  of  complex 
graphical  information  such  as  virtual  scenarios,  medical 
simulators,  or  simple  anatomical  data  over  the  Internet  using 
modems  and  low-band  networks,  remains  still  an  unresolved 
technical  challenge.  Broad-band  networks  such  as  the  Internet 
2  initiative  may  be  a  solution,  but  remote  applications  are 
most  needed  in  areas  of  low  resources,  so  work  on  efficient 
transmission  of  medical  information  over  the  Internet  is  clue 
if  these  applications  are  to  be  available  to  everyone. 

V.  Conclusion 

We  propose  in  this  paper  a  complete  methodology  for  the 
creation,  manipulation  and  transmission  of  3D  anatomical 
models  starting  from  stacks  of  medical  images.  We  use  well- 
established  segmentation  and  reconstruction  algorithms 
previoulsy  developed  by  the  authors  and  focus  on  the 
manipulation  and  transmission  of  the  models.  We  use 
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